Which is More Suitable for Chinese Word Segmentation, the Generative Model or the Discriminative One?

PACLIC 23 / City University of Hong Kong / 3-5 December 200

GPSM: A Generalized Probabilistic Semantic Model for Ambiguity Resolution

In natural language processing, ambiguity resolution is a central issue, and can be regarded as a preference assignment problem. In this paper, a Generalized Probabilistic Semantic Model (GPSM) is proposed for preference computation. An effective semantic tagging procedure is proposed for tagging semantic features. A semantic score function is de- rived based on a score function, which integrates lexical, syntactic and semantic prefer- ence under a uniform formulation. The se- mantic score measure shows substantial im- provement in structural disambiguation over a syntax-based approach